Hierarchical Skills and Cognitive Architectures

In this paper, we examine approaches to representing and utilizing hierarchical skills within the context of a cognitive architecture. We review responses to this issue by three established frameworks – ACT-R, Soar, and Prodigy – then present an alternative we have developed within Icarus, another candidate architecture. Unlike most earlier systems, Icarus lets skills refer directly to their subskills and communicate within a single recognize-act cycle. This assumption has implications for the number of cycles required to complete complex tasks. We illustrate our approach with the domain of multi-column subtraction, then discuss related methods and directions for future work in this area. Introduction and Overview Human skills are organized in a hierarchical fashion. There seems to be general agreement with this claim, as it is consistent not only with experimental findings about the execution and acquisition of skills, but also with introspection about our everyday behavior. Upon request, most people can describe their complex skills at successive levels of aggregation, whether these involve how they drive to work each day, how they cook a meal, or how they write a technical article. What remains an open question is how we should model such skill hierarchies in computational terms. Alternative approaches to modeling cognition encode the notion of hierarchy in distinct ways that have different implications for performance and learning. The most interesting positions are those which are embedded in theories of the human cognitive architecture, such as Soar (Laird et al., 1987), ACT-R (Anderson, 1993), and EPIC (Kieras & Meyer, 1997). These frameworks make strong commitments to both the mental representations of knowledge and to the processes that operate on them. In the pages that follow, we consider the challenge of modeling hierarchical skills within a unified theory of cognition. We begin with a brief review of three such architectures and their responses to this issue, then turn to Icarus, an alternative framework that approaches hierarchical skills from a different perspective. The key issue involves whether one can traverse levels of a hierarchy within a single cognitive cycle. Our illustrative example comes from a familiar cognitive skill that has a hierarchical organization – multi-column subtraction – but we also consider other models that incorporate multi-level skills. We conclude by discussing related work and our plans to extend the architecture’s capabilities. Previous Research on Hierarchical Skills A cognitive architecture (Newell, 1990) specifies the infrastructure for an intelligent system, indicating those aspects of a cognitive agent that remain unchanged over time and across different application domains. Many proposals for the human cognitive architecture take the form of a production system, which stores long-term knowledge as a set of condition-action rules, encodes short-term elements as declarative list structures, and relies on a recognize-act cycle that matches against, and executes rules that alter, the contents of short-term memory. Soar, ACT-R, and EPIC are all examples of the production-system paradigm, although other architectural frameworks are also possible. Despite the general agreement that cognitive skills are organized hierarchically, there exist different ways to implement this basic idea. Within a production-system architecture, the most natural scheme involves communication between skills and their subskills through the addition of elements to short-term memory. For instance, ACT-R achieves this effect using production rules that match against a generalized goal structure in their condition sides, such as the desire to prove that two triangles are congruent, and, upon firing, create new subgoals, such as the desire to prove that two lines have the same length. This approach requires the explicit addition of goal elements to short-term memory, since this is the only mechanism through which other production rules can be accessed. Soar takes a somewhat different approach to encoding hierarchical skills that relies on multiple problem spaces. For instance, Jones and Laird (1997) report a detailed model of flying an aircraft in combat training scenarios. This system organizes its capabilities in a hierarchical manner, with each node implemented as a Soar problem space with associated states, operators, and goals. To invoke a lower-level problem space, the system adds a new element to short-term memory that refers to that space, and it must take similar action in order to exit. The details differ from those in ACT, but the passing of messages through short-term memory is similar. The Prodigy architecture (Minton et al., 1989) produces cognitive behavior in yet another manner. The system represents knowledge about actions as Stripslike operators, and the central module utilizes meansends analysis to decompose problems into subproblems. This gives Prodigy the ability to generate hierarchical structures dynamically for each problem it encounters, and it can use control rules to select among candidate operators, states, and goals that determine the decompositions. However, these hierarchical structures do not remain in memory after a problem has been solved; instead, the system stores its experience in generalized control rules that let it reconstruct them for new problems. Again, this requires adding explicit elements to shortterm memory that serve as mediators. Thus, despite their diversity, these three frameworks share a basic assumption about how communication occurs among skills and subskills. This tenet has implications for the cognitive behavior of systems cast within them, including whether intermediate results are available, the time required to execute complex skills, and the effects of learning on hierarchical structure. We will see shortly that another response to this issue is possible. Hierarchical Skills in Icarus Icarus (Choi et al., in press) is a cognitive architecture that shares some central features with its predecessors. For instance, it relies on symbolic list structures to encode information, and it makes a clear distinction between long-term and short-term memories, with the former storing generic knowledge and the latter containing specific beliefs. Moreover, Icarus assumes a recognizeact cycle, which relies on matching patterns in long-term memory against elements in the short-term store, to determine behavior over time. The framework also comes with a programming formalism for building knowledgebased systems. However, Icarus has other characteristics that distinguish it from earlier frameworks in significant ways. One such feature is its commitment to embodied cognition, which requires that all symbols in Icarus programs be grounded in sensori-motor primitives. A second key assumption is that each long-term memory structure may have not only a symbolic description but also an associated numeric function that computes its value as a function of sensory attributes. A third novel feature is that Icarus contains distinct long-term memories for skills, which store its knowledge about action, and for concepts, which specify its knowledge about states and relations. Yet another contribution lies in the strong correspondence between long-term and short-term memory, specifically that every element in the latter must be an instance of some structure in the former. A fifth important assumption, which is our focus here, is that hierarchical structures play a central role in cognition. Although production systems and related architectures allow hierarchies, many do not encourage them, and we maintain that Icarus supports them in a sense that is stronger and more plausible psychologically than do frameworks like ACT and Soar. To understand this claim, we must first examine the architecture’s representation for skills and the relations among them. Some recent architectural variants, like ACT-R/PM and EPIC-Soar, incorporate sensori-motor modules, but these were grafted onto systems based on theories of human problem solving, whereas Icarus included them from the outset. Icarus skills bear a strong resemblance to production rules, but they have an even closer kinship to the operators in Prodigy and Strips. Each skill has a name, arguments, and some optional fields. These include: • a :start field, which encodes the situation that must hold to initiate the skill; • a :requires field, which must be satisfied throughout the skill’s execution across multiple cycles; and • an :effects field, which specifies the desired situation the skill is intended to achieve. For example, Table 1 shows a complete set of Icarus skills for the domain of multi-column subtraction, including borrow, which has the objective of getting ?digit1 to be greater than ?digit2. This skill can start only if ?digit2 is greater than ?digit1 and if a third element, ?digit3, is nonzero. Moreover, its execution requires that ?digit1 be above ?digit2, that ?digit3 be in the top row, and that ?digit3 be left of ?digit1. In addition, each Icarus skill includes another field that specifies how to decompose it into subskills or actions. This may be either: • an :ordered field, which indicates the agent should consider the components in a specific order; • an :unordered field, which identifies a choice among skill components; or • an :actions field, in which a primitive skill specifies one or more actions that are directly executable. For example, borrow states that one should invoke decrement on one digit and call add-ten on another, in that order. These are both primitive skills that play the same role as Strips operators in a traditional planning system, with their :start fields serving as preconditions and their :effects fields specifying the desired results of execution. The table also clarifies that Icarus can specify multiple ways to decompose each skill in this manner, much as a Prolog program can include more than one Horn clause with the same head. Different decompositions of a given skill must have the same name, number of arguments, and effects. However, they can differ in their start conditions, requirements, and subskills. The skill borrow has two such expansions, one for borrowing from nonzero elements and another for borrowing across zero, which involves a recursive call to itself. Each skill decomposition may also include a numeric function that encodes the utility expected if it executes this decomposition. This function is specified by a :percepts field that matches against the values of objects’ attributes and a :value field that states an arithmetic combination of these quantities. The expected utility for a skill decomposition is a linear function of the numeric descriptors matched by that skill. Such functions are not required when the available skills specify deterministic behavior, as do those in the table, but we have used them in other domains and we mention them here for completeness. We should note that Icarus also organizes its longterm conceptual memory in a hierarchy, with higher-level concepts being defined in terms of lower-level ones, which Table 1: Icarus skills for multi-column subtraction.